Performance Gain with Variable Chunk Size in GFS-like File Systems

We have designed and implemented Tianwang File System(TFS), which is a distributed file system much like Google file system(GFS). The system has its origins in our Tianwang search engine and web mining research work. Our system has the same assumptions and the same architectures with GFS. But the key design choice that the chunk size is variable lets our system to adopt simpler system interactions which significantly improves the performance of the record append operation. In this paper, we discuss many aspects of our design which are different from GFS, and verify their pros and cons by performance experiments. The experiment results shows that the utilization ratio of our record append operation excels GFS by 25%. And the throughput of record append of TFS is also several times better.